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Abstract 

This study examined the increase from 2007-2009 in Kentucky’s fourth- grade NAEP reading 
scores. This increase was notable because it came after a fifteen- year period of no change. This 
analysis of change included composite and subscale — Reading Literary Experience and Reading 
to Gain Information— scores. Differences in composite average scale scores by Teacher Quality 
variables were explored. Gender was included as a variable for analysis of both composite and 
subscale scores as gender gaps have been persistent in Kentucky’s reading assessments. This 
secondary analysis used NAEP Data Tool with additional calculations of effect size (Cohen’s d). 
There was a significant (p.<. 05) increase in Kentucky 4 th -Grade Reading scores from 2007 and 
2009 with small effect size (<7= 0.12). Statistically significant (p.c. 05) differences by Teacher 
Quality variables were: (1) reading scores of students taught by teachers with masters’ degrees 
were higher than those taught by teachers with bachelors’ degrees, (2) reading scores of students 
taught by teachers with 0-4 years of teaching experience were lower than the scores of students 
taught by teachers with five or more years of teaching experience and (3) there was no difference 
in reading scores of students taught by teachers self-identified as Highly Qualified Teachers and 
the scores students taught by teachers not self- identified as Highly Qualified Teachers. The 
scores of female students on composite scale scores were statistically significantly (p. <05) 
higher than the scores of male students in both 2007 (d= 0.21) and 2009 (<7=0.21). Females 
students’ scores on the Reading Literary Experience subscale were also higher than male 
students; scores in both 2007 (<7=0.27) and 2009 (d=0.28). On the Reading to Gain Information 
subscale in 2007, average scale scores of female students were higher than those of male students 
(<7 = 0.14). It is important to note that there was no difference by gender on the Reading to Gain 



Information subscale in 2009. 
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Kentucky’s Teacher Quality Measures and Fourth-Grade Reading Achievement: A Secondary 

Analysis of 2002-2009 NAEP Data 

On March 24, 2010, the Kentucky Department of Education (KDE) reported that 
Kentucky was one of three states that had a statistically significant increase in fourth-grade grade 
reading scores from 2007-2009 on the National Assessment for Educational Progress (NAEP). 
Additionally, NAEP fourth and eighth grade reading average scale scores for Kentucky 
surpassed the national average in 2009 (KDE, 2010). This was welcome news in the 
Commonwealth of Kentucky as there had been no increase in Kentucky’s NAEP fourth-grade 
reading average scale scores for the prior fifteen years (Klecker, 2008). 

Since 1999, NAEP average scale scores have been used in validation studies of 
Kentucky’s accountability testing system (Koger, Thacker, Koger, & Deatz, 2003). Between 
1992 and 1998, the NAEP average scale score on Kentucky’s fourth-grade reading assessment 
increased only five points (Educational Trust, 2002). There were no statistically significant 
(alpha =.05) differences in the NAEP fourth grade reading average scale scores from 1992-2007 
(Klecker, 2008). 

Purpose of the Study 

The purpose of this study was to examine the increase in Kentucky’s fourth" grade 
reading scores on the National Assessment of Educational Progress (NAEP) from 2007 to 2009 
after a fifteen-year period of no change. First, a secondary data analysis examined differences in 
fourth-grade reading scores by Kentucky’s Teacher Quality Measures. Next the researcher 
examined NAEP 2007 and 2009 fourth-grade composite scores, and subscale scores— reading for 
literary experience and reading for information. The final analysis of the 2007-2009 NAEP data 
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examined gender differences as this persistent gap in reading achievement for fourth-grade 
Kentucky students has persisted. 

Teacher quality measures. The No Child Left Behind Act (NCLB) (2002) included 
teacher quality as a major factor that impacts student learning. Teacher quality was defined by 
NCLB as credentials: (1) major/minor in subject area of teaching assignment, (2) completion of 
required degree, (3) certification in area of teaching assignment, (4) years of teaching 
experience.). Kentucky uses the NCLB 2002 definitions (KDE, 2006). 

Researchers have found that the quality of the teacher in the classroom was 
the most important schooling factor predicting student achievement (e.g., Darling-Hammond, 
2000; Darling-Hammond, and Sykes, 2003; Hanhushek, Kain, & Rivkin, 1998).Wayne and 
Youngs (2003) found strong links between the NCLB-defined teacher quality variables and 
student achievement. Klecker (2007) found statistically significant differences between teacher 
quality measures and students’ 2005 NAEP national eighth-grade mathematics achievement. 

The National Assessment of Educational Progress (NAEP) has since 1969, been the only 
nationally representative and continuing assessment of what America's students know in various 
subject areas. The No Child Left Behind Act (2002) required participation of all school receiving 
Title I money from federal funds. This requirement has enlarged the data base. In 2005, teacher 
questionnaire data were added to include the NCLB-defined teacher quality variables. 

However, in 2009, the teacher-quality question regarding the teacher’s, “Type of Certification” 
was not available. However, a new question was added, “This school year, are you a Highly 
Qualified Teacher (HQT) according to your state's requirements?” (teacher-reported) 

ID: T096601, Values: Yes, At least 1 subject, No” (NCES, 2010). Results from the analysis by 
Teacher Quality variables are presented in Tables 2 through 5. 
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Framework for fourth-grade reading assessment. The framework for NAEP reading 
was changed for the 2009 assessments. NCES (2010) presented the following information about 
the changes: 

The 1992-2007 framework 

The 1992-2007 framework specified that the NAEP assessment should measure three 
contexts for reading: reading for literary experience, reading for information, and reading 
to perform a task (reading to perform a task at grades 8 and 12 only). In addition to 
reading within different contexts, NAEP reading comprehension questions were 
developed to engage the different approaches that readers may take in the process of 
trying to understand what is being read. The table below outlines the major aspects of the 
framework.... (p. 1) 

Comparison of the 1992-2007 and 2009 frameworks 

The framework for the 2009 NAEP reading assessment replaces a framework developed 
for the 1992 assessment. Compared to the previous framework, the 2009 reading 
framework includes more emphasis on literary and informational texts, a redefinition of 
reading cognitive processes, a new systematic assessment of vocabulary knowledge, and 
the addition of poetry to grade 4. . .(p. 1). 

NAEP Fourth-Grade Reading assessment data were available for the analyses presented in this 
paper as a composite score and two subscales scores: Gain information scale and Literary 
experience scale. NCES described the scales as: 

Literary and informational texts 

The NAEP reading framework specifies the use of both literary and informational texts. 
Literary texts include three types at each grade: fiction, literary nonfiction, and poetry. 
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Informational texts include three broad categories: exposition; argumentation and 
persuasive text; and procedural text and documents. The inclusion of distinct text types 
recognizes that students read different texts for different purposes.... (p. 1). 

The results of the secondary analysis of the data from the 2007 and 2009 Fourth-Grade Reading 
assessments Kentucky are presented by composite and subscale average scale scores in Tables 1. 
6. And 7. 

Examining the gender gap in Kentucky’s fourth-grade reading NAEP scores. 

The analyses by subcategories by gender informed the areas for growth for continued 
improvement in Kentucky’s reading achievement. Tables 8 through 10 present analyses by 
gender with strong evidence that the gender gap in Kentucky’s fourth grade reading scores on the 
NAEP persist. 

Method 

Data Source 

The National Assessment of Educational Progress (NAEP) has since 1969, been the only 
nationally representative and continuing assessment of what America's students know in various 
subject areas. The No Child Left Behind Act (2002) required participation of all school receiving 
Title I money from federal funds. This requirement has enlarged the data base. In 2005, teacher 
questionnaire data were added to include the NCLB-defined teacher quality variables (NCES, 
2010 ). 

Data Analysis 

The author of this study attended training to analyze the unique NAEP data in June 2008. 
During this visit, the data analyses of NAEP of 2007 data were performed. For 
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the follow-up of the 2002-2007 and analysis of the teacher quality variables in 2007 data. For 
additional analysis of 2007 compared with 2009 NAEP scores, the researcher used —as advised 
by NAEP training staff— the NAEP Data Tool available on the NAEP website. [IRB approval 
was obtained for this study.] 

Table 1. Kentucky’s 2007 and 2009 Fourth-Grade Reading NAEP Scores 



Year 


Average Scale Score 


SD 


2009 


226 


32 


2007 


222 


34 



The average scale score for 2009 (226) was statistically significantly higher p =.043 than the 
average scale score for 2007 (225), with an effect size of <i=0.12. 

Table 2. Major or Minor in Elementary Education by Fourth- Grade Reading Scores 2009 



Variable 


Average Scale Score 


SD 


Major in El. Ed. 


226 


32 


Minor/special emphasis 


225 


29 


Neither 


225 


34 



Tested with an alpha level of .05, there were no statistically significant differences in the average 
scale scores of the variable categories. 
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Table 3. Highest Academic Degree by Kentucky’s Fourth-Grade Reading Scores 2009 



Variable 


Average Scale Score 


SD 


Bachelor’s Degree 


221 


33 


Master’s Degree 


228 


31 


Education Specialist 


223 


32 



Tested with alpha =.05, there was a statistically significant differences between the 
Average scale score of students whose teachers held a Bachelor’s Degree (221) and the average 
scale score of students whose teacher held a Master’s Degree (228) with an effect size, p =.013, 
d= 0.22. 

Table 4 Years Taught Elementary or Secondary by Fourth-Grade Reading Scores 2009 



Variable Level 


Average Scale Score 


SD 


Cohen’s d 


0-4 years 


219 


32 




5-9 years 


226 


32 


d= 0.22 


10-19 years 


227 


31 


J=0.25 


20+ years 


232 


31 


d=0Al 



Tested with an alpha =.05, the average scale score of the 4 th grade students whose 
teachers had 0-4 years of teaching experience (219) was statistically significantly lower than the 
average scale scores of students of teachers in the other levels of teaching experience. Effect 
sizes between 0-4 level of teaching experience and the other levels of years of teaching 
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experience are presented in Table 4. After their teachers had 5 years or more teaching 
experience, there were no statistically significant differences in their students’ average scale 
scores.. 

Table 5. Highly Qualified Teacher (HQT) This Year by Students’ 2009 4 th Grade 
Reading Scores 



Variable 


Average Scale Score 


SD 


Yes 


226 


32 


No 


223 


34 



The average scale score of the 4 th grade students whose teachers reported that they had 
met their state’s standard as a Highly Qualified Teacher (HQT) in 2009, was not statistically 
significantly different from that of the students whose teachers were not HQTs. p. = ,297 . 

Table 1 presented the results of the Kentucky NAEP 4 th Grade Reading average 
scale scores for 2007 and 2009 composite (total scale) score. Tables 6 and 7 below presents 
analyses of differences in the subscales. 

Table 6. Reading to Gain Information Subscale of Kentucky’s NAEP 4 th Grade Reading 
Assessment for 2007 and 2009 



Variable 


Average Scale Score 


SD 


2009 


225 


33 


2007 


222 


36 
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Tested with alpha =.05, the average scale score of students in 2009 was statistically 
significantly higher than the average scale score of students in 2007, p = .017, <7=0.09. 

Table 7. Reading Literary Experience Subscale on Kentucky NAEP 4 th Grade Reading 
Assessment 2007 and 2009 



Variable 


Average scale score 


SD 


2009 


226 


34 


2007 


223 


35 



Tested with alpha =.05, there were no statistically significant differences between the 
average scale score on the Reading Literary Experience Subscale by Year, p = .184. 

Table 8. NAEP Kentucky 4 th Grade Reading Composite Scale Scores by Gender by Year 



Variable 


Average scale score 




SD 






Male 


Female 


Male 


Female 


2009 


222 


229 


37 


34 


2007 


219 


226 


35 


32 



Tested with alpha=.05, the average scale score of female students on the composite in 
2009 (229) was statistically significantly higher than the average scale score of male students 
(222), p =..001, <7 = 0.20. In 2007, the average scale score of female students (226) was 
statistically significantly higher than the average scale score of male students (219), p = .001, 



<7=0.21. 
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Table 9. Reading to Gain Information Subscale by Gender by Year 



Variable 


Average scale score 




SD 






Male 


Female 


Male 


Female 


2009 


224 


227 


33 


33 


2007 


219 


224 


37 


35 



Tested with alpha = .05, there was no statistically significant difference by gender in the 
2009 average scale scores on the Reading to Gain Information subscale, p = .061. In 2007, 
average scale score of females (224) was statistically significantly higher than the males’ average 
scale score (219), p = .018, d = 0.14. 

Table 10. Reading Literary Experience Subscale by Gender by Year 



Variable 


Average scale score 




SD 






Male 


Female 


Male 


Female 


2009 


221 


230 


33 


33 


2007 


218 


228 


37 


35 



Tested with alpha = .05. In 2009, the average scale score of female students (230) was 
Statistically significantly higher than male students’ average scale score (221), p = .001, d = 
0.27. In 2007, female students’ average scale score (228) was statistically significantly higher 
than male students’ average scale score (218), p = .001, d=0.28. 




KENTUCKY’S TEACHER QUALITY MEASURES AND FOURTH-GRADE 12 

Conclusions 

The statistically significant increase in Kentucky’s fourth grade reading average scale 
score on the NAEP should be celebrated. Although the increase is small, Kentucky’s fourth and 
eighth grade reading scores surpassed the national average. The analyses by Teacher Quality 
Measures indicate: 

• the reading scores of fourth-grade students taught by teachers with masters’ degrees are 
higher than those of teachers with bachelors’ degrees— and Ed.S. degrees had little 
impact on students’ reading scores. 

• The reading scores of students who were taught by teachers with 0-4 years of teaching 
experience were statistically significantly lower than the scores of students who were 
taught by teachers with five or more years of teaching experience. After five years of 
teaching experience by the classroom teacher, the scores of students did not significantly 
increase. 

• There was no difference in the fourth-grade reading scores of students who were taught 
by teachers self-identified as Highly Qualified Teachers and the reading scores of 
students who were taught by teachers who were not Highly Qualified Teachers. 

Gender differences. Female students’ average scale score was higher than male 

students’ average scale score on the composite scale for both 2007 and 2009 and on the Reading 
Literary Experience subscale for both 2007 and 2009. 

The good news in closing the gender gap in reading scores: There was no statistically 
significant difference by gender in the 2009 average scale scores on the Reading to Gain 



Information subscale, p = .061. 
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As Kentucky moves to a national standards-based education in reading and mathematics 
(Senate Bill 1), instruction in Kentucky classrooms in fourth-grade reading is likely to be re- 
aligned. Clear progress is being made in reading instruction. 
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